1. Field of the Invention
The present invention relates to a musical scale recognition method and an apparatus thereof, and more specifically, to a musical scale recognition method and an apparatus thereof for comparing an input audio signal with a predetermined musical note.
2. Description of the Prior Art
Such a kind of a conventional musical scale recognition apparatus made measurements of a frequency component employing a Fast Fourier Transformation (FFT) in regard to an input audio signal, and carried out a musical scale recognition on the basis of the measurement result thereof.
However, there needed to be a microprocessor or a DSP (digital signal processor) with a high processing capability in order to analyze all frequency components included in the audio signal in real time because the Fourier transformation had to be done at high speed.
Therefore, a primary object of the present invention is to provide a musical scale recognition method and an apparatus thereof that even a microprocessor with a low processing capability can perform a musical scale recognition in real time.
A first musical scale recognition method according to the present invention, comprises following steps of (a) converting an input analog audio signal into digital data D by sampling the audio signal at constant intervals C; (b) deriving sin xcfx89t and cos xcfx89t (xcfx89 is an angular velocity in correspondence to an observed frequency f) based upon the observed frequency f and a time t; (c) calculating a cumulative value As to find a coefficient of a Fourier sine series by performing an operation of an equation (1); (d) calculating a cumulative value Ac to find a coefficient of a Fourier cosine series by performing an operation of an equation (2); (e) calculating a frequency power spectrum effective value A by performing an operation of an equation (3); (f) evaluating a component of the frequency f included in an analog audio signal on the basis of the numeric value A; and (g) renewing the time t by performing an operation of an equation (4).
As←As+Dxc2x7sin xcfx89txe2x80x83xe2x80x83(1)
Ac←Ac+Dxc2x7cos xcfx89txe2x80x83xe2x80x83(2)
A←{square root over (As2+Ac2)}xe2x80x83xe2x80x83(3)
t←t+Cxe2x80x83xe2x80x83(4)
A second musical recognition method according to the present invention, comprises following steps of (a) converting an input analog audio signal into digital data D by sampling the audio signal at constant intervals C; (b) deriving sin xcfx89t and cos xcfx89t (xcfx89 is an angular velocity in correspondence to an observed frequency f) based upon the observed frequency f and a time t; (c) a calculating a cumulative value As to find a coefficient of a Fourier sine series by performing an operation of the above equation (1); (d) calculating a cumulative value Ac to find a coefficient of a Fourier cosine series by performing an operation of the above equation (2); (e) calculating a frequency power spectrum effective value A by performing an operation of a below equation (5); (f) evaluating a component of the frequency f included in the analog audio signal on the basis of the numeric value A; and (g) renewing the time t by performing an operation of the above equation (4).
xe2x80x83A←As2+Ac2xe2x80x83xe2x80x83(5)
A first musical recognition apparatus according to the present invention, comprises an analog/digital converting means which converts an input analog audio signal into a digital data D by sampling the audio signal at constant intervals C; a deriving means which derives sin xcfx89t and cos xcfx89t (xcfx89 is an angular velocity in correspondence to an observed frequency f) based upon the observed frequency f and a time t; a first operating means which calculates a cumulative value As to find a coefficient of a Fourier sine series by performing an operation of an equation (1); a second operating means which calculates a cumulative value Ac to find a coefficient of a Fourier cosine series by performing an operation of an equation (2); a third operating means which calculates a frequency power spectrum effective value A by performing an operation of an equation (3); an evaluating means which evaluates a component of the frequency f included in the analog audio signal on the basis of the numeric value A; and a renewing means which renews the time t by performing an operation of above equation (4).
A second musical recognition apparatus according to the present invention, comprises an analog/digital converting means which converts an input analog audio signal into a digital data D by sampling the audio signal at constant intervals C; a deriving means which derives sin xcfx89t and cos xcfx89t (xcfx89 is an angular velocity in correspondence to an observed frequency f) based upon the observed frequency f and a time t; a first operating means which calculates a cumulative value As to find a coefficient of a Fourier sine series by performing an operation of the above equation (1); a second operating means which calculates a cumulative value Ac to find a coefficient of a Fourier cosine series by performing an operation of the above equation (2); a third operating means which calculates a frequency power spectrum effective value A by performing an operation of the above equation (5); an evaluating means which evaluates a component of the frequency f included in the analog audio signal on the basis of the numeric value A; and a renewing means which renews the time t by performing an operation of the above equation (4).
A third musical scale recognition apparatus according to the present invention, comprises a BGM reproducing means which reproduces a karaoke BGM on the basis of musical score data; musical score data storing means which stores musical score data and musical scale data having an exemplary melody for singing included in synchronous with the musical score data; a reading means which reads the musical scale data from the musical score data storing means at a time t; a setting means which sets a frequency of the musical scale data read by the reading means to an observed frequency f; a musical scale recognition means which performs a musical scale recognition by using any one of the above musical scale recognition methods; and an outputting means which outputs an evaluation result by the evaluating means.
A fourth musical scale recognition apparatus according to the present invention, comprises, a BGM reproducing means which reproduces a karaoke BGM on the basis of musical score data; a musical score data storing means which stores musical score data and musical scale data having an exemplary melody for singing included in synchronous with the musical score data; a reading means which reads musical scale data from the musical score data storing means at a time t; a setting means which sets a frequency of the musical scale data read by the reading means to an observed frequency f0, a frequency of a musical scale one octave below the musical scale data read by the reading means to an observed frequency f1, and a frequency of a musical scale one octave above the musical scale data read by the reading means to an observed frequency f2; a musical scale recognition means which carries out a musical scale recognition by using the above described musical scale recognition methods; and an outputting means which outputs an evaluation result by the evaluation means.
A fifth musical scale recognition apparatus according to the present invention, comprises a musical scale recognition means which sequentially carries out a musical scale recognition of an analog audio signal by using any one of the above musical scale recognition methods; a comparing means which compares a changing pattern of the musical scale recognized by the musical scale recognition means with a predetermined musical phrase; and a first operating means which performs a predetermined operation brought into correspondence with this relevant musical phrase when the changing pattern of the musical scale recognized by the musical scale recognition means becomes coincident with the predetermined musical phrase as a result of a comparison by the comparing means.
A sixth musical scale recognition apparatus according to the present invention, comprises a musical note data storing means which stores musical scale data of each musical note of a musical phrase; a pointer which points one of musical note data included in the musical note data storing means; a musical note data reading means which reads the musical scale data of the musical note pointed by the pointer from the musical note data storing means; a setting means which sets a frequency of the musical scale data read by the musical note data reading means to an observed frequency f; a musical scale recognition means which sequentially performs a musical scale recognition of an analog audio signal by using any one of the above described musical scale recognition methods; a comparing means which compares a degree of a frequency component of the frequency f included in the analog audio signal with a predetermined threshold value; a pointer manipulating means which, as a result of a comparison by the comparing means, increments the pointer when the degree of the frequency component of the frequency f included in the analog audio signal is larger than the predetermined threshold value and points at the musical scale data of the musical note at the forefront of the musical phrase by the pointer when the degree of the frequency component of the frequency f included in the analog audio signal is less than the predetermined threshold value; and a first operating means which performs a predetermined operation brought into correspondence to the relevant musical phrase when a value of the pointer exceeds a position of the musical scale data of the musical note at the end of the musical phrase.
In the first invention, provided that digital data having the input analog audio signal converted by an analog/digital converter is D, a frequency (musical scale) of a musical sound to be recognized is f, and a current time is t, calculations are made as to a cumulative value As to find a coefficient of a Fourier sine series of the audio signal on the basis of the frequency f and the digital data D, a cumulative value Ac to find a coefficient of a Fourier cosine series of the audio signal on the basis of the frequency f and the digital data D, a frequency power spectrum effective value of the audio signal on the basis of the cumulative value As and the cumulative value Ac. Then, it is evaluated to what extent the component of the observed frequency f is included in the analog audio signal on the basis of the numeric value A.
In a preferred embodiment, the numeric value A is evaluated after the input analog audio signal is corrected in correspondence to a level of an amplitude of the input analog audio signal.
In a further preferred embodiment, there exist a plurality of the observation frequencies (f0, f1 . . . , fNxe2x88x921: N indicates the number of units of the frequencies to be simultaneously observed), and it is evaluated to what extent the component of the respective observation frequencies is included in the analog audio signal.
In the second invention, a level of consistency between the singing voices and an exemplary melody is evaluated in such a manner that singing voices sung along a karaoke BGM are subjected to a musical scale recognition on the basis of the exemplary melody for a singing.
More specifically, the BGM is reproduced on the basis of the musical score data, and the voices in tune with the BGM are input. The musical score data includes the musical scale data, i.e. the exemplary melody for singing in synchronous with the musical score data. When the BGM of the time t is being reproduced, the musical scale data at the time t is read from the musical score data.
Then, a musical scale recognition is applied to the singing voices at the time t on the basis of the frequency f of the read-out musical scale data, and evaluations are applied to an extent of the component of the frequency f included in the singing voices, i.e. an extent of consistency between the singing voices and the melody. It is possible to appropriately make a music scale recognition even though a reproduction pitch of the karaoke BGM is changed because the musical scale data is in synchronous with the musical score data.
There are cases of being sung on a musical scale one octave below or above the exemplary melody for singing. Therefore in the third invention, the musical scale recognition is applied to the singing voices sung along the BGM on the basis of a melody one octave below and above the exemplary melody for singing. In regard to the musical scale recognition, the musical scale recognition method is adopted as claimed in any of claims 1 to 4.
In the fourth invention, the musical scale recognition of the analog audio signal is successively applied in order to determine whether or not the analog audio signal is coincident with a predetermined musical phrase. Upon being coincident, a predetermined operation previously brought into correspondence to the relevant musical phrase is performed. In regard to the musical scale recognition, a musical scale recognition method is adopted as claimed in any of claims 1 to 4.
In a preferred embodiment, it is determined whether or not every single musical note in the musical phrase is included in the analog audio signal, and once it is determined a first sound is included as a result of the musical scale recognition, it is then determined whether or not a second note is further included. If and when the sound of any musical note is not included in the analog audio signal, the musical scale recognition is once again performed to determine whether or not the first sound is included in the analog audio signal. Subsequently, in a musical phrase including musical notes in N units, it is determined that the analog audio signal is coincident with the relevant musical phrase if and when it is determined that an N-th sound is included in the analog audio signal.
It is noted that when the musical scale recognition is applied to the analog audio signal by the N-th sound of the musical phrase, a musical scale recognition of the analog audio signal in correspondence to the N-th sound is performed during the length of musical note of the N-th sound.
In a further preferred embodiment, when the analog audio signal is coincident with the predetermined musical phrase, a code previously brought into correspondence to the relevant musical phrase is transmitted by blinking an infrared light-emitting element, for example.
In addition, a device which has received the transmitted code causes a light-emitting element, e.g. LED to blink in a pattern previously brought into correspondence to the code, and output from the speaker an audio signal having a content brought into correspondence to the code, and so forth on.
According to the present invention, a musical scale recognition of the input voices is performed by a simple processing, i.e. comparing a specific frequency component expected to be input with the input voices.
Therefore, it is possible to implement a device which carries out a musical scale recognition in real time by using a microprocessor with a low processing capability.
The above described objects and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.